{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "06c68eed-3391-49b0-8dd9-381e818b2c75",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "# 1、numpy的高级特性\n",
    "## 1.1、numpy的广播\n",
    "### 1.1.1、np.newaxis(插轴操作)\n",
    "np.newaxis在切片位置插入一个新轴(新维度)，长度为 1，不拷贝数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "46fdce49-0a59-4687-b785-2c481ff6359f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1 2 3]\n",
      "[[1 2 3]]\n",
      "[[1]\n",
      " [2]\n",
      " [3]]\n",
      "****************************************************************************************************\n",
      "[[1 2 3]]\n",
      "[[1]\n",
      " [2]\n",
      " [3]]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "x = np.array([1, 2, 3])   # shape: (3,)\n",
    "print(x)\n",
    "print(x[None,:])# 表示没有列只有行\n",
    "print(x[:,None])# 表示只有列，没有行\n",
    "print('*'*100)\n",
    "x1 = np.expand_dims(x,axis=0)\n",
    "print(x1)\n",
    "x2 = np.expand_dims(x,axis=1)\n",
    "print(x2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "76457c39-edd3-4abc-a14d-50df81ef6e58",
   "metadata": {},
   "source": [
    "### 1.1.2、广播基础\n",
    "广播是 NumPy 在按元素运算时，自动把形状兼容（某轴为 1）的一方“虚拟拉伸”来对齐形状的机制——不真的复制数据。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "47da11cd-52be-4b51-98be-e9e2da7e9ac9",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "a = np.array([1,2,3])\n",
    "b = np.array([3,4,5])\n",
    "#数组之间是可以直接运算的，前提就是要对齐\n",
    "c = a+b\n",
    "print(a,a.shape)\n",
    "print(c,c.shape)\n",
    "#二维数组之间也可以直接进行运算的\n",
    "a = np.array([[1,2,3],[4,5,6]])\n",
    "b = np.array([[1,1,1],[2,2,2]])\n",
    "#直接对位运算\n",
    "print(a.shape,b.shape,c.shape)\n",
    "print(a+b)\n",
    "#c是一维数组，此时进行运算之后会自动广播到每一行\n",
    "print(a+c)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ce1ee23a-762d-4939-8f4a-da8a7908ce73",
   "metadata": {},
   "source": [
    "### 1.1.3、广播的对齐逻辑\n",
    "右对齐，对齐逐轴比较：每一位要么相等，要么其中一个是 1\n",
    "举例:(3,2)可以和(3,1)进行广播，也可以和(1,2)进行广播"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "04e0098d-d0b7-4d33-9d80-94616bd382f9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[1. 1. 1. 1.]\n",
      " [1. 1. 1. 1.]]\n",
      "[[4 5 6 7]\n",
      " [2 3 4 5]]\n",
      "[[2. 3. 4. 5.]\n",
      " [2. 3. 4. 5.]]\n",
      "[[4. 4. 4. 4.]\n",
      " [2. 2. 2. 2.]]\n",
      "[[2. 3. 4. 3.]\n",
      " [4. 5. 6. 4.]]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "#对齐维度就可以进行运算\n",
    "a = np.ones((2,4))\n",
    "print(a)\n",
    "b = np.array([[1,2,3,4]])\n",
    "c = np.array([[3],[1]])\n",
    "#(1,4)(2,5)可以广播\n",
    "print(b+c)\n",
    "d = np.array([[1,2,3,2],[3,4,5,3]])\n",
    "e = np.array([[1,2],[2,2]])#不能进行运算\n",
    "print(a+b)\n",
    "print(a+c)\n",
    "print(a+d)\n",
    "# print(a+e)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b0e4346e-d32c-40d2-8dbc-5790e5a7eb4f",
   "metadata": {},
   "source": [
    "如果两个数组的维度不同，会自动左补，如果是一个(3,)的一维数组会自动转换为(1,3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "9c60899a-82ca-4bfd-a0e1-459c8c90634e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[1. 1. 1.]\n",
      " [1. 1. 1.]] (2, 3)\n",
      "[1 3 3] (3,)\n",
      "[[2. 4. 4.]\n",
      " [2. 4. 4.]]\n",
      "[1 2] (2,)\n",
      "[[2. 2. 2.]\n",
      " [3. 3. 3.]]\n"
     ]
    }
   ],
   "source": [
    "a = np.ones((2,3))\n",
    "print(a,a.shape)\n",
    "b = np.array([1,3,3]) #b的维度是(3,)\n",
    "# #numpy在两个集合进行运算的时候，如果维度不同会进行左补。\n",
    "# #b的维度是3,会补成(1,3)这样就可以和a维度对齐进行运算\n",
    "print(b,b.shape)\n",
    "print(a+b)\n",
    "c = np.array([1,2])\n",
    "print(c,c.shape) #(2,)补之后是(1,2)\n",
    "# # print(a+c) #会报错，因为(2,3)无法和(1,2)相加，没有对齐\n",
    "#可以把c转换为2,1就可以\n",
    "print(c[:,None]+a)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "55936519-1f97-4ff3-819d-44a80ce96912",
   "metadata": {},
   "source": [
    "### 1.1.4、广播的练习\n",
    "#### 练习 A｜判断与心算（不写代码）\n",
    "\n",
    "1) 判断是否可广播；若可，给出**结果形状**：  \n",
    "   a. `(5,4)` 和 `(4,)`  \n",
    "   b. `(5,4)` 和 `(5,)`  \n",
    "   c. `(2,3)` 和 `(1,3)`  \n",
    "   d. `(2,3)` 和 `(2,1)`  \n",
    "   e. `(6,)` 和 `(2,3)`  \n",
    "   f. `(3,1)`、`(1,4)` 与 `(3,4)` 三者一起\n",
    "\n",
    "2) 下列哪句能实现**列广播**（`A` 形状 `(m,n)`，`col` 为一维或二维）？  \n",
    "   a. `A + col`（其中 `col.shape == (m,)`）  \n",
    "   b. `A + col[:, None]`  \n",
    "   c. `A + col.reshape(1, m)`  \n",
    "   d. `A + col.reshape(m, 1)`\n",
    "***\n",
    "#### 练习 B｜改错题（让它能跑）\n",
    "```python\n",
    "A = np.ones((2,3))\n",
    "b = np.array([10, 20])      # 目标：按列加到 A 上\n",
    "A + b                        # ← 修改为可运行的写法\n",
    "```\n",
    "***\n",
    "```python\n",
    "A = np.arange(6).reshape(2,3)\n",
    "c = np.array([1,2,3])\n",
    "np.concatenate([A, c], axis=1)  # 想把 c 当行拼到右侧，改法？\n",
    "```\n",
    "***"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "7e4c6bd7-85a2-4fba-8584-753c59b41236",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[11., 11., 11.],\n",
       "       [21., 21., 21.]])"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A = np.ones((2,3))\n",
    "b = np.array([10, 20])      # 目标：按列加到 A 上\n",
    "A + b[:,None]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "f5c10d6f-b79c-420b-a8a5-c60a46da7af7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0, 1, 2],\n",
       "       [3, 4, 5],\n",
       "       [1, 2, 3]])"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A = np.arange(6).reshape(2,3)\n",
    "c = np.array([1,2,3])\n",
    "np.concatenate((A, c[None,:]), axis=0)  # 想把 c 当行拼到右侧，改法？"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cf3b838f-9f96-4d1c-88c1-ab24f364f188",
   "metadata": {},
   "source": [
    "#### 练习 C｜编码题（直接写一行/几行）\n",
    "\n",
    "- 两两和矩阵：给 a(50,)、b(60,)，用广播得到 (50,60) 的两两和。\n",
    "- 两两差矩阵：给 x(100,)，得到 D(i,j)=|x_i-x_j| 的 (100,100)。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "b7128d5b-4d70-4d83-8d7b-a947ceb7c98b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[6. 6. 6. ... 6. 6. 6.]\n",
      " [6. 6. 6. ... 6. 6. 6.]\n",
      " [6. 6. 6. ... 6. 6. 6.]\n",
      " ...\n",
      " [6. 6. 6. ... 6. 6. 6.]\n",
      " [6. 6. 6. ... 6. 6. 6.]\n",
      " [6. 6. 6. ... 6. 6. 6.]]\n",
      "[[0. 0. 0. ... 0. 0. 0.]\n",
      " [0. 0. 0. ... 0. 0. 0.]\n",
      " [0. 0. 0. ... 0. 0. 0.]\n",
      " ...\n",
      " [0. 0. 0. ... 0. 0. 0.]\n",
      " [0. 0. 0. ... 0. 0. 0.]\n",
      " [0. 0. 0. ... 0. 0. 0.]] (100, 100)\n"
     ]
    }
   ],
   "source": [
    "# 两两和矩阵：给 a(50,)、b(60,)，用广播得到 (50,60) 的两两和。\n",
    "import numpy as np\n",
    "a = np.ones(50)\n",
    "b = np.full(60,5)\n",
    "print(a[None,:]+b[:,None])\n",
    "# print(a+b)\n",
    "#两两差矩阵：给 x(100,)，得到 D(i,j)=|x_i-x_j| 的 (100,100)。\n",
    "X = np.ones(100)\n",
    "D = X[None,:]-X[:,None]\n",
    "print(D,D.shape)\n",
    "#按列 Z-score：X(m,n)，把每列标准化到均值 0、方差 1（不能 for）。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "06614ee7-42ce-4601-83cc-36f0d786096d",
   "metadata": {},
   "source": [
    "### 1.1.5、应用举例\n",
    "#### **应用举例1:Z-score讲解**\n",
    "按列 Z-score：X(m,n)，把每列标准化到均值 0、方差 1（不能 for）。主要作用是统一尺度：不同量纲（cm、kg、￥…）放到同一数量级，便于比较，公式如下所示:\n",
    "$z=\\dfrac{x-\\mu}{\\sigma}\\;\\Rightarrow\\; z=0\\;\\Leftrightarrow\\; x=\\mu \\;(\\sigma\\ne 0)$\n",
    "\n",
    "假设有三科成绩\n",
    "- 语文：0–100 分\n",
    "- 数学：0–150 分\n",
    "- 物理：0–200 分\n",
    "创建4位同学的成绩（X 的形状是 (4,3)：4 行学生 × 3 列科目）："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "9fb7d02f-78ab-4910-8eb8-6da4dd844ed2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 75.  105.  132.5]]\n",
      "[[  5.   15.   27.5]\n",
      " [ 15.  -15.  -32.5]\n",
      " [ -5.   30.   47.5]\n",
      " [-15.  -30.  -42.5]]\n",
      "[[11.18033989 23.71708245 38.32427429]]\n",
      "****************************************************************************************************\n",
      "[[ 0.4472136   0.63245553  0.71756088]\n",
      " [ 1.34164079 -0.63245553 -0.84802649]\n",
      " [-0.4472136   1.26491106  1.23942334]\n",
      " [-1.34164079 -1.26491106 -1.10895772]]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "stus = np.array([\n",
    "    [80,120,160],\n",
    "    [90,90,100],\n",
    "    [70,135,180],\n",
    "    [60,75,90]\n",
    "])\n",
    "#先按列取平均值,第二个参数表示保持维度\n",
    "mu = stus.mean(axis=0,keepdims=True)\n",
    "print(mu)\n",
    "#计算出我的成绩离平均值的距离\n",
    "print(stus-mu)\n",
    "std = stus.std(axis=0,keepdims=True)\n",
    "print(std)\n",
    "print('*'*100)\n",
    "#Z-score完成量纲标准化\n",
    "print((stus-mu)/std)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "88fedc37-8edf-4e6a-af2e-47d9cb6c4ae0",
   "metadata": {},
   "source": [
    "#### **应用举例2:归一化的处理**\n",
    "归一化是将数据压缩在0-1的范围内，也是一种统一量纲的方法，和zscore的区别在于，这个能够维持相对比例关系，二zscore会保留相对偏差关系使用的公式如下\n",
    "\n",
    "$x'=\\frac{x - x_{\\min}}{x_{\\max} - x_{\\min}}$\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "948f46e8-f9fe-4b29-af58-fbd44382869e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[0.66666667 0.75       0.77777778]\n",
      " [1.         0.25       0.11111111]\n",
      " [0.33333333 1.         1.        ]\n",
      " [0.         0.         0.        ]]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "stus = np.array([\n",
    "    [80,120,160],\n",
    "    [90,90,100],\n",
    "    [70,135,180],\n",
    "    [60,75,90]\n",
    "])\n",
    "x_min = stus.min(axis=0,keepdims=True)\n",
    "x_max = stus.max(axis=0,keepdims=True)\n",
    "X = (stus-x_min)/(x_max-x_min)\n",
    "#得到的是相对的比例关系\n",
    "print(X)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0a3f525a-2554-46fb-bcf1-5953bbbbb3b8",
   "metadata": {},
   "source": [
    "#### **应用举例3:掩码替换**\n",
    "**“掩码替换”** 它背后的意义，和数据清洗、异常值处理、特征工程都有直接关系。\n",
    "\n",
    "**掩码**：用布尔索引筛出需要修改的元素；\n",
    "\n",
    "**替换**：用对应列的均值替代（通过广播实现列方向对齐）。\n",
    "\n",
    "假设我们认为成绩中 0–20 属于异常低分（可能是缺考、录入错误、测量噪声等），那么可以把 所有落在 [0,20] 区间 的数值用该科目的均值来替代。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "9a16f390-4d37-4a29-b8a1-7264b8a43f08",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 48.75  90.   112.5 ]]\n",
      "[[False False False]\n",
      " [False False  True]\n",
      " [ True False False]\n",
      " [ True  True False]]\n",
      "[[ 48.75  90.   112.5 ]\n",
      " [ 48.75  90.   112.5 ]\n",
      " [ 48.75  90.   112.5 ]\n",
      " [ 48.75  90.   112.5 ]]\n",
      "****************************************************************************************************\n",
      "[20. 15. 10. 15.]\n",
      "[112.5   48.75  48.75  90.  ]\n",
      "****************************************************************************************************\n",
      "[[ 80.   120.   160.  ]\n",
      " [ 90.    90.   112.5 ]\n",
      " [ 48.75 135.   180.  ]\n",
      " [ 48.75  90.    90.  ]]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np \n",
    "A = np.array([\n",
    "    [80,120,160],\n",
    "    [90,90,20],\n",
    "    [15,135,180],\n",
    "    [10,15,90]\n",
    "], dtype=float)\n",
    "a,b = 0,20\n",
    "col_mean = A.mean(axis=0,keepdims=True)\n",
    "mask = (A>=a)&(A<=b)\n",
    "print(col_mean)\n",
    "print(mask)\n",
    "#需要把col_mean扩展成A的大小\n",
    "T = col_mean.repeat(A.shape[0],axis=0)\n",
    "print(T)\n",
    "print('*'*100)\n",
    "print(A[mask])\n",
    "print(T[mask])\n",
    "A[mask] = T[mask]\n",
    "print('*'*100)\n",
    "print(A)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5b310091-3017-4fe2-8fd5-43cbef8f40ac",
   "metadata": {},
   "source": [
    "#### **应用举例4：图像信息预处理**\n",
    "图像通道标准化（仅 2D/3D 简化版）：img(H,W,3) 与 mean(3,)、std(3,)，完成 (img-mean)/std。这个问题的意义其实和 Z-score 标准化几乎一模一样，只不过对象从“表格中的特征列”变成了“图像的三个颜色通道”。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "48e27e9c-5b47-444e-affa-1c71e9c72151",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(2, 2, 3)\n",
      "(3,) (3,)\n",
      "[[[-0.5        -1.         -1.5       ]\n",
      "  [-0.75       -1.33333333 -2.        ]]\n",
      "\n",
      " [[ 0.75        0.33333333  0.5       ]\n",
      "  [ 2.          2.          3.        ]]]\n",
      "[[[-0.40542855  0.06512605  0.46135076]\n",
      "  [-0.57667609 -0.10994398  0.28705882]]\n",
      "\n",
      " [[ 0.45080914  0.76540616  1.15851852]\n",
      "  [ 1.30704684  1.6407563   2.02997821]]]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "# 模拟一张小图像（高2宽2，每像素3通道RGB）\n",
    "img = np.array([\n",
    "    [[100,120,130],\n",
    "     [ 90,110,120]],\n",
    "    [[150,160,170],\n",
    "     [200,210,220]]\n",
    "], dtype=float)  # shape (2,2,3)\n",
    "print(img.shape)\n",
    "# 通道均值和标准差\n",
    "mean = np.array([120, 150, 160])\n",
    "std  = np.array([40, 30, 20])\n",
    "print(mean.shape,std.shape)\n",
    "img_nor = (img - mean)/std\n",
    "print(img_nor)\n",
    "\n",
    "#基于深度学习的值\n",
    "mean = [0.485, 0.456, 0.406]\n",
    "std  = [0.229, 0.224, 0.225]\n",
    "img_nor2 = (img / 255.0 - mean) / std\n",
    "print(img_nor2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e31176f5-d6e1-47c3-8896-ce5fa2242d6f",
   "metadata": {},
   "source": [
    "#### **应用举例5:加权处理**\n",
    "X(m,n) 与 权重 w(n,)，返回 y(m,) = Σ_j X[i,j]*w[j]（用广播或矩阵乘），加权的意义就是为不同的特征值确定重要性，依然可以使用学生成绩的例子来进行"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "id": "02a1e3d8-a006-450d-946e-7cd39b81ef6a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[32.  36.  48. ]\n",
      " [36.  27.  30. ]\n",
      " [28.  40.5 54. ]\n",
      " [24.  22.5 27. ]]\n",
      "[116.   93.  122.5  73.5]\n",
      "[[116. ]\n",
      " [ 93. ]\n",
      " [122.5]\n",
      " [ 73.5]] (4, 1)\n",
      "[116.   93.  122.5  73.5] (4,)\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "stus = np.array([\n",
    "    [80,120,160],\n",
    "    [90,90,100],\n",
    "    [70,135,180],\n",
    "    [60,75,90]\n",
    "])\n",
    "w = np.array([0.4,0.3,0.3])\n",
    "print(stus*w)\n",
    "print(np.sum(stus*w,axis=1))\n",
    "#相当于矩阵乘法\n",
    "S1 = stus@w[:,None]\n",
    "print(S1,S1.shape)\n",
    "#numpy的矩阵乘法会自动处理\n",
    "S2 = stus@w\n",
    "print(S2,S2.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5a93e460-74d0-4fa3-9bcb-70136196075c",
   "metadata": {},
   "source": [
    "## 1.2、文件的生成和读取\n",
    "### 1.2.1、写csv文件"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "d175af7e-6d89-4806-9dea-06860b2b493e",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[49 76 55]\n",
      " [40 89 99]\n",
      " [68 65 69]\n",
      " [88 69 89]\n",
      " [98 48 49]\n",
      " [40 82 80]\n",
      " [94 76 91]\n",
      " [56 76 87]\n",
      " [51 94 64]\n",
      " [83 98 73]]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "np.random.seed(10)#固定随机种子，生成的数一样\n",
    "scores = np.random.randint(40,100,size=(10,3))\n",
    "print(scores)\n",
    "np.savetxt('scores.csv',scores,fmt='%d',delimiter=',',header='语文,数学,物理',comments='',encoding='utf-8-sig')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1a068ee6-48bc-47d6-a24d-32203298d215",
   "metadata": {},
   "source": [
    "### 1.2.2、读取csv文件"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "b276ef4b-83aa-49e5-9eab-6e7c3ac3de29",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[49 76 55]\n",
      " [40 89 99]\n",
      " [68 65 69]\n",
      " [88 69 89]\n",
      " [98 48 49]\n",
      " [40 82 80]\n",
      " [94 76 91]\n",
      " [56 76 87]\n",
      " [51 94 64]\n",
      " [83 98 73]]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "scores = np.loadtxt('scores.csv',delimiter=',',skiprows=1,dtype=int,encoding='utf-8-sig')\n",
    "print(scores)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "47c65447-3b2b-44e0-896e-b166c82c41bd",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "## 1.3、numpy的数据分析于矩阵运算\n",
    "### 1.3.1、统计函数与分布分析\n",
    "创建一个学生成绩来完成处理"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "0c351645-3a27-44af-b0c6-c14ceb769c6f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[49 76 55]\n",
      " [40 89 99]\n",
      " [68 65 69]\n",
      " [88 69 89]\n",
      " [98 48 49]\n",
      " [40 82 80]\n",
      " [94 76 91]\n",
      " [56 76 87]\n",
      " [51 94 64]\n",
      " [83 98 73]]\n",
      "****************************************************************************************************\n",
      "各科均值: [66.7 77.3 75.6] [66.7 77.3 75.6]\n",
      "各科标准差： [21.32158531 13.9645981  15.58974022]\n",
      "各科最高分： [98 98 99]\n",
      "各科最低分： [40 48 49]\n",
      "各科极差： [58 50 50]\n",
      "全班均值： 73.2\n",
      "总体标准差： 17.865049678072545\n",
      "50%的百分比 62.0\n",
      "四分位数（语文）： [49.5  62.   86.75]\n",
      "[[49.5  70.75 65.25]\n",
      " [62.   76.   76.5 ]\n",
      " [86.75 87.25 88.5 ]]\n",
      "62.0\n",
      "[62.  76.  76.5]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "scores = np.loadtxt('scores.csv',delimiter=',',skiprows=1,dtype=int,encoding='utf-8-sig')\n",
    "print(scores)\n",
    "print('*'*100)\n",
    "print('各科均值:',np.mean(scores,axis=0),scores.mean(axis=0))\n",
    "print(\"各科标准差：\", np.std(scores, axis=0))\n",
    "print(\"各科最高分：\", np.max(scores, axis=0))\n",
    "print(\"各科最低分：\", np.min(scores, axis=0))\n",
    "print(\"各科极差：\", np.ptp(scores, axis=0))\n",
    "#总体统计\n",
    "print(\"全班均值：\", np.mean(scores))\n",
    "print(\"总体标准差：\", np.std(scores))\n",
    "print(\"50%的百分比\",np.percentile(scores[:,0],50))#语文中有62%的人分数低于50\n",
    "print(\"四分位数（语文）：\", np.percentile(scores[:,0], [25,50,75]))\n",
    "#按照科类统计\n",
    "print(np.percentile(scores,[25,50,75],axis=0))\n",
    "#统计每一科的中位数，不受极端值影响，更能代表“典型水平”\n",
    "print(np.median(scores[:,0]))\n",
    "print(np.median(scores,axis=0))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8e09cddc-a879-446e-96e4-67f2bb5b82a4",
   "metadata": {},
   "source": [
    "### 1.3.2、统计函数练习\n",
    "- 基础础练习\n",
    "    - 求每门课的平均分、最高分、最低分。\n",
    "    - 计算每门课的极差、标准差。\n",
    "    - 找出全班语文成绩的中位数与四分位数。\n",
    "- 提高练习\n",
    "    - 判断哪一门课“波动最大”（标准差最大）。\n",
    "    - 对语文成绩排序。\n",
    "    - 求出全班综合平均成绩，并计算每位学生与平均分的差值（Z-score）。\n",
    "    - 对三门课的平均分进行加权求和，算出总评（复习前一节内容）并且添加到最后。\n",
    "    - 使用广播方法让每位学生成绩减去各科平均值（均值中心化)。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "3e4bdc7d-2f63-48be-ab92-2484a2793bff",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[49 76 55]\n",
      " [40 89 99]\n",
      " [68 65 69]\n",
      " [88 69 89]\n",
      " [98 48 49]\n",
      " [40 82 80]\n",
      " [94 76 91]\n",
      " [56 76 87]\n",
      " [51 94 64]\n",
      " [83 98 73]]\n",
      "****************************************************************************************************\n",
      "[66.7 77.3 75.6]\n",
      "[98 98 99]\n",
      "[40 48 49]\n",
      "极差: [58 50 50]\n",
      "标准差: [21.32158531 13.9645981  15.58974022]\n",
      "中位数: 62.0\n",
      "四分位数: [49.5  62.   86.75]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "scores = np.loadtxt('scores.csv',delimiter=',',skiprows=1,dtype=int,encoding='utf-8-sig')\n",
    "print(scores)\n",
    "print('*'*100)\n",
    "#求每门课的平均分、最高分、最低分。\n",
    "print(scores.mean(axis=0))\n",
    "print(scores.max(axis=0))\n",
    "print(scores.min(axis=0))\n",
    "#计算每门课的极差、标准差。\n",
    "print(\"极差:\",np.ptp(scores,axis=0))\n",
    "print(\"标准差:\",scores.std(axis=0))\n",
    "#找出全班语文成绩的中位数与四分位数。\n",
    "yw = scores[:,0]\n",
    "print(\"中位数:\",np.median(yw))\n",
    "print(\"四分位数:\",np.percentile(yw,[25,50,75]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "6dd44878-962e-42bf-afc6-13a2b614dd5b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[49 76 55]\n",
      " [40 89 99]\n",
      " [68 65 69]\n",
      " [88 69 89]\n",
      " [98 48 49]\n",
      " [40 82 80]\n",
      " [94 76 91]\n",
      " [56 76 87]\n",
      " [51 94 64]\n",
      " [83 98 73]]\n",
      "****************************************************************************************************\n",
      "[21.32158531 13.9645981  15.58974022]\n",
      "0\n",
      "[[49]\n",
      " [40]\n",
      " [68]\n",
      " [88]\n",
      " [98]\n",
      " [40]\n",
      " [94]\n",
      " [56]\n",
      " [51]\n",
      " [83]]\n",
      "****************************************************************************************************\n",
      "[1 5 0 8 7 2 9 3 6 4]\n",
      "[[40 89 99]\n",
      " [40 82 80]\n",
      " [49 76 55]\n",
      " [51 94 64]\n",
      " [56 76 87]\n",
      " [68 65 69]\n",
      " [83 98 73]\n",
      " [88 69 89]\n",
      " [94 76 91]\n",
      " [98 48 49]]\n",
      "****************************************************************************************************\n",
      "73.2\n",
      "[60.         76.         67.33333333 82.         65.         67.33333333\n",
      " 87.         73.         69.66666667 84.66666667]\n",
      "[11.5758369  25.78113005  1.69967317  9.20144916 23.33809475 19.34482417\n",
      "  7.87400787 12.83225104 18.00617178 10.27402334]\n",
      "[[-0.95025527  1.38218948 -0.43193421]\n",
      " [-1.39637013  0.50424477  0.89212536]\n",
      " [ 0.39223227 -1.37281295  0.98058068]\n",
      " [ 0.6520712  -1.41282093  0.76074973]\n",
      " [ 1.41399717 -0.72842279 -0.68557439]\n",
      " [-1.41295331  0.75817007  0.65478324]\n",
      " [ 0.88900089 -1.3970014   0.50800051]\n",
      " [-1.32478705  0.23378595  1.0910011 ]\n",
      " [-1.03668158  1.35138849 -0.31470691]\n",
      " [-0.16222142  1.29777137 -1.13554995]]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "scores = np.loadtxt('scores.csv',delimiter=',',skiprows=1,dtype=int,encoding='utf-8-sig')\n",
    "print(scores)\n",
    "print('*'*100)\n",
    "# 判断哪一门课“波动最大”（标准差最大）。\n",
    "#1、得到每一列的标准差\n",
    "stds = scores.std(axis=0)\n",
    "print(stds)\n",
    "#2、得到下标\n",
    "max_idx = np.argmax(stds)\n",
    "print(max_idx)\n",
    "#3、输出列\n",
    "print(scores[:,[0]])\n",
    "# 对语文成绩排序。\n",
    "print('*'*100)\n",
    "sort_idx = np.argsort(scores[:,0],axis=0)\n",
    "print(sort_idx)\n",
    "print(scores[sort_idx])\n",
    "# 求出全班综合平均成绩，并计算每位学生与平均分的差值（Z-score）。\n",
    "print('*'*100)\n",
    "#综合平均成绩\n",
    "print(scores.mean())\n",
    "#每个学生的平均成绩\n",
    "stu_mean = scores.mean(axis=1)\n",
    "print(stu_mean)\n",
    "#每个学生的标准差\n",
    "stu_std = scores.std(axis=1)\n",
    "print(stu_std)\n",
    "#stu_mean是(10,)需要升维\n",
    "print((scores-stu_mean[:,None])/stu_std[:,None])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "ffc7ad44-0b72-4068-8f8d-81ef40c4040c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[49 76 55]\n",
      " [40 89 99]\n",
      " [68 65 69]\n",
      " [88 69 89]\n",
      " [98 48 49]\n",
      " [40 82 80]\n",
      " [94 76 91]\n",
      " [56 76 87]\n",
      " [51 94 64]\n",
      " [83 98 73]]\n",
      "****************************************************************************************************\n",
      "[[49.  76.  55.  61. ]\n",
      " [40.  89.  99.  71.4]\n",
      " [68.  65.  69.  67. ]\n",
      " [88.  69.  89.  80.6]\n",
      " [98.  48.  49.  68.2]\n",
      " [40.  82.  80.  64.8]\n",
      " [94.  76.  91.  86.2]\n",
      " [56.  76.  87.  70.2]\n",
      " [51.  94.  64.  70.8]\n",
      " [83.  98.  73.  87. ]]\n",
      "[66.7 77.3 75.6]\n",
      "[[-17.7  -1.3 -20.6]\n",
      " [-26.7  11.7  23.4]\n",
      " [  1.3 -12.3  -6.6]\n",
      " [ 21.3  -8.3  13.4]\n",
      " [ 31.3 -29.3 -26.6]\n",
      " [-26.7   4.7   4.4]\n",
      " [ 27.3  -1.3  15.4]\n",
      " [-10.7  -1.3  11.4]\n",
      " [-15.7  16.7 -11.6]\n",
      " [ 16.3  20.7  -2.6]]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "scores = np.loadtxt('scores.csv',delimiter=',',skiprows=1,dtype=int,encoding='utf-8-sig')\n",
    "print(scores)\n",
    "print('*'*100)\n",
    "#对三门课的平均分进行加权求和，算出总评（复习前一节内容）并且添加到最后。\n",
    "weight = np.array([0.4,0.4,0.2])\n",
    "res=  scores@weight;\n",
    "st = np.append(scores,res[:,None],axis=1)\n",
    "print(st)\n",
    "#使用广播方法让每位学生成绩减去各科平均值（均值中心化）。\n",
    "#1、求出每科的平均值\n",
    "m1 = scores.mean(axis=0)\n",
    "print(m1)\n",
    "#(10,3)和(1,3)\n",
    "print(scores-m1)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.13.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
